feat(stt): warm audio pipeline on prepare and retain between recordings#247
Open
chriswritescode-dev wants to merge 5 commits into
Open
feat(stt): warm audio pipeline on prepare and retain between recordings#247chriswritescode-dev wants to merge 5 commits into
chriswritescode-dev wants to merge 5 commits into
Conversation
- New VoiceActivityDetector class: adaptive noise floor, per-frame RMS
classification, trailing-silence endpointing, configurable thresholds
- AudioWorkletProcessor computes RMS in existing flush loop and posts
{samples, rms} instead of a bare Int16Array
- AudioRecorder integrates VAD via handleVadFrame() in both worklet and
ScriptProcessor paths; auto-calls stop() on trailing silence; silence
gate in processRecording() fires onNoSpeech instead of sending empty
audio to the transcription API (fixes Whisper hallucination bug)
- useSTT wires setOnNoSpeech to reset isProcessing/isRecording cleanly
without surfacing an error
- computeRms() exported helper for ScriptProcessor fallback path
stopRecording() called recorder.stop() before setIsProcessing(true). For a silent take the recorder synchronously fires onNoSpeech (which sets processing false), so the subsequent setIsProcessing(true) won and the voice overlay was stuck in the processing state forever. Set processing before stop() so the recorder callback (onNoSpeech or onDataAvailable) has the final say on the processing flag.
…icks Two remaining stuck paths after the silent-stop fix: - processRecording() early-returned without firing onNoSpeech when a take had no captured audio (e.g. a quick start/stop under one worklet flush, totalSamples === 0). Now any non-emitting path (empty audio or no detected speech) fires onNoSpeech, while a genuine abort still returns silently. - stopRecording() set isProcessing true even when the recorder was no longer recording (double-stop or post auto-stop), where stop() skips processRecording entirely and nothing clears the flag. Guard the external path on recorder state before entering processing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
prepare()method toAudioRecorderto warm the audio context and worklet before first recording. Retain the audio pipeline between recordings (only the mic track is stopped after each use). Full resources are released viadispose()when STT is disabled or the component unmounts.AudioRecorder.prepare()to pre-warm audio context and workletAudioRecorder.dispose()for full cleanup of all resourcescleanup()→cleanupRecording(closeAudioContext)to distinguish partial vs full cleanupdispose()during asyncstart()prevents entering recording stateuseSTTto usedisposeAudioRecorderon unmount and timeoutdocs/features/stt.mdFiles
docs/features/stt.mdfrontend/src/hooks/useSTT.tsfrontend/src/hooks/useSTT.test.tsxfrontend/src/lib/audioRecorder.tsfrontend/src/lib/audioRecorder.test.ts